{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 奇异值分解"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "奇异值分解是一类矩阵分解方法，它能够将一个大矩阵转化成几个小矩阵相乘的形式，这些小矩阵中蕴含着大矩阵的很多重要特征。正如我们要记住一张陌生的脸总是先记住这张脸的种种特征一样，奇异值分解能够帮助我们从矩阵中提取特征，而这些特征正是我们“认识”这个矩阵的重要桥梁。在人工智能非常流行的今天，奇异值分解也不仅仅只是一个数学问题，而被更广泛地应用到图片压缩，图片去噪，数据降维，推荐系统等等方面中去。本文将着重介绍奇异值分解的背景知识以及应用。\n",
    "\n",
    "## 矩阵坐标变换"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "奇异值分解可以非常简明地使用数学语言描述：\n",
    "\n",
    ">对于每一个线性映射T: Kn → Km，T把Kn的第i个基向量映射为Km的第i个基向量的非负倍数，然后将余下的基向量映射为零向量。对照这些基向量，映射T就可以表示为一个非负对角阵。\n",
    "\n",
    "然而这样的概括太过抽象，因此笔者想从比较直观的几何意义来阐明奇异值分解的意义。各位读者可能本科的时候就学过线性代数，线性代数的入门知识就是矩阵乘法，那时候老师只会告诉我们矩阵的乘法法则，可是矩阵乘法的意义特别是物理意义是什么呢？这一小节笔者将尝试从几何的角度阐述矩阵乘法。\n",
    "\n",
    "首先用一句话概括矩阵乘法的几何意义：矩阵乘法就是将m空间下的一组坐标映射到n空间下坐标的过程。为了能够直观地观察，这里的空间坐标变换全部都限定为二维平面空间。假定现在平面坐标系上有四个点:\n",
    "\n",
    "$$\n",
    "p_{1}=\\begin{bmatrix}\n",
    "0\\\\0 \n",
    "\\end{bmatrix} \n",
    "p_{2}=\\begin{bmatrix}\n",
    "0\\\\1 \n",
    "\\end{bmatrix} \n",
    "p_{3}=\\begin{bmatrix}\n",
    "1\\\\1\n",
    "\\end{bmatrix} \n",
    "p_{4}=\\begin{bmatrix}\n",
    "1\\\\0 \n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在坐标系上这几个点构成的图形如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAACjtJREFUeJzt3U+InPd9x/HPtyghjoTJHyVrnEN1LU1iGq+RDTWsYl/cQsglick9SnvqIYReSo4pOQQTKJTo1EBoRA8hVxMdBmxiOV35GNJDqe1LBU1txV0TlVj+5bCTjiti7Z9npGf03dcLBLO7s/N8+TL75tl5NGyNMQLA/e2P5h4AgOnEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAZO3asDnT17dpw7d+5eHe59vf322zl9+vTcY2wEu9hnDyt2sbIpu7h27dqvxhifOOh+9yzm586dy+7u7r063PtaLBbZ2dmZe4yNYBf77GHFLlY2ZRdV9dph7udlFoAGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGJsW8qs5X1c+q6oWqem5dQwFwNFPPzF9L8vkxxpNJPllVn1nDTAAc0aS/ATrGuP6eD99JcmvaOAAcR40xpj9I1WeT/P0Y4y9v+/zFJBeTZGtr69HLly9PPtZUe3t7OXPmzNxjbAS72GcPK3axsim7uHDhwrUxxvZB95sc86r6WJKfJPnybWfq/8/29vbY3d2ddKx12JS/uL0J7GKfPazYxcqm7KKqDhXzqRdATyX5YZJv3inkANxdUy+AfinJY0m+U1WLqnpiDTMBcERTL4D+KMmP1jQLAMfkTUMADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtDApJhX1cNV9UpV3ayqU+saCoCjmXpm/kaSp5JcXcMsABzTpLPpMcbNJDerak3jAHAcXhrhRPvnl1/PD17+Tf7x316ae5SN8Ccf/m125h6CY7mrMa+qi0kuJsnW1lYWi8XdPNyh7O3tbcQcm8Aukh+8/Ju8/tatJDfmHmV2r//Pu/nvD48T/5z4vfvt5+OuxnyMcSnJpSTZ3t4eOzs7d/Nwh7JYLLIJc2wCu8jyjPxGnv/bZ+YeZXZf+f5LuXHjxol/Tvze/fbzMfV/s3ygqq4keSTJ81V1fj1jAXAUUy+A/jbJ02uaBYBj8qYhgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGpgc86p6rqpeqKrvrWMgAI5uUsyr6nNJTo8xnkzywap6bD1jAXAUpyZ+/xNJrixvX0nyeJJ/nfiYcM+8/B9vJEm+8v2XZp5kfr/4z7fy8ANzT8FxTY35R5L8+/L2r5P86Xu/WFUXk1xMkq2trSwWi4mHm25vb28j5tgEdrFy48aNuUeY3cMPJH/28VueE0v328/H1JjfSPLg8vaDy4//zxjjUpJLSbK9vT12dnYmHm66xWKRTZhjE9hF8uqOPbyXXazcb7uYegH0pSRPLW8/neTqxMcD4BgmxXyM8UqSm1X1QpJ3xxg/X89YABzF1JdZMsb4m3UMAsDxedMQQANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0cOyYV9UzVfXLqnpxnQMBcHRTzsyvJnlkXYMAcHynjvuNY4w3k6Sq1jcNAMfiNXOABmqMcec7VD2U5PJtn74+xnh2+fUXxxh//j7fezHJxSTZ2tp69PLl2x/m3tvb28uZM2fmHmMj2MU+e1ixi5VN2cWFCxeujTG2D7rfgTE/8AHuEPP32t7eHru7u5OOtQ6LxSI7Oztzj7ER7GKfPazYxcqm7KKqDhXzKf+bZbuqriT5dFVdqaoPHfexAJhmygXQ3SRPr3EWAI7JBVCABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaEHOABsQcoAExB2hAzAEaOHbMq+prVXV1+e+r6xwKgKOZcmb+0zHG40meTPKNNc0DwDEcO+ZjjFeXN99Jcmst0wBwLDXGmPYAVX+d5KNjjG//ga9dTHIxSba2th69fPnypGOtw97eXs6cOTP3GBvBLvbZw4pdrGzKLi5cuHBtjLF90P0OjHlVPZTk9gpfH2M8W1Xnk/xdki+OMe54dr69vT12d3cPmueuWywW2dnZmXuMjWAX++xhxS5WNmUXVXWomJ866A5jjOtJdv7AAT6V5LtJvnBQyAG4u6ZcAP1Wkq0kP66qRVU9sKaZADiiA8/M388Y4+vrHASA4/OmIYAGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGDvyDzms7UNV/JXntnhzszs4m+dXcQ2wIu9hnDyt2sbIpu/jjMcYnDrrTPYv5pqiq3cP8peuTwC722cOKXazcb7vwMgtAA2IO0MBJjPmluQfYIHaxzx5W7GLlvtrFiXvNHKCjk3hmDtDOiYt5VX2tqq4u/3117nnmVlXPVNUvq+rFuWeZQ1U9V1UvVNX35p5lblX1cFW9UlU3q+rU3PPMparOV9XPls+L5+ae57BOXMyT/HSM8XiSJ5N8Y+5hNsDVJI/MPcQcqupzSU6PMZ5M8sGqemzumWb2RpKnsv+cOMleS/L55fPik1X1mbkHOowTF/MxxqvLm+8kuTXjKBthjPHmGON/555jJk8kubK8fSXJ4zPOMrsxxs0xxptzzzG3Mcb1McbN5Yf3TSdOXMzf46+S/GTuIZjVR5K8tbz96yQfnXEWNkxVfTbJ2THGL+ae5TDavi5WVQ8luXzbp6+PMZ6tqvNJ/iLJF+/9ZPO40z7mmGdD3Ejy4PL2g8uPIVX1sST/kOTLc89yWG1jPsa4nmTn9s9X1aeSfDfJF8YY98WvT+vwfvs44V5K8vUk/5Lk6ST/NOs0bITlxd8fJvnm8ufmvnASX2b5VpKtJD+uqkVVPTD3QHOqqu2qupLk01V1pao+NPdM98oY45UkN6vqhSTvjjF+PvdMc6qqDyyfC48keX75G+xJ9KUkjyX5zrIRT8w90GF40xBAAyfxzBygHTEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGfgdZ9XfaqVAVXQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import matplotlib as mpl\n",
    "\n",
    "mpl.rcParams[\"xtick.labelsize\"] = 8\n",
    "mpl.rcParams[\"ytick.labelsize\"] = 8\n",
    "\n",
    "v = np.asmatrix([[0, 0], [0, 1], [1, 1], [1, 0], [0, 0]], dtype=np.float32)\n",
    "fig = plt.figure(\"test\")\n",
    "ax = fig.add_subplot(111)\n",
    "ax.grid(True)\n",
    "ax.plot(v[:, 0], v[:, 1])\n",
    "plt.xlim(-2.5, 2.5)\n",
    "plt.ylim(-2.5, 2.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "之所以要使用一个矩形来表示是因为这样能够更清楚地显示出矩阵坐标变换对图形的影响。我们都知道：标准坐标系使用标准正交基向量表示就是x轴和y轴方向上的单位向量。在这里有必要解释一下“正交基向量”这个概念：通常我们表示一个坐标系时是选取该坐标系下的一组正交向量来表示，例如二维坐标系那么选取两个相互正交的向量就能够表示，如果是三维坐标系就需要用三个相互正交的向量来表示。而且为了表示方便，通常情况下正交的基向量是单位向量。当然，使用非单位正交向量也能够表示一个坐标系。上图所示的标准坐标系使用标准正交基向量表示就是：\n",
    "\n",
    "$$W = (w_{1}, w_{2}) =  \\begin{bmatrix}\n",
    "1 & 0\\\\ \n",
    "0 & 1\n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到以上向量两两正交且向量的模为1，则一个点在基向量上的投影可以表示为：\n",
    "\n",
    "$$p_{i} = W^{T}p_{i}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的四个点构成的向量集合可以表示为：\n",
    "\n",
    "$$P=(p_{1},p_{2},p_{3},p_{4})=\\begin{bmatrix}\n",
    "0 &  0&  1& 1\\\\ \n",
    "0 &  1&  0& 1\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "则四个点在两个正交坐标基向量上的投影为：\n",
    "\n",
    "$$P'=W^{T}P=(w_{1}, w_{2})^{T}(p_{1},p_{2},p_{3},p_{4})=\\begin{bmatrix}\n",
    "1 & 0\\\\ \n",
    "0 & 1\n",
    "\\end{bmatrix}\\begin{bmatrix}\n",
    "0 &  0&  1& 1\\\\ \n",
    "0 &  1&  0& 1\n",
    "\\end{bmatrix}=\\begin{bmatrix}\n",
    "0 &  0&  1& 1\\\\ \n",
    "0 &  1&  0& 1\n",
    "\\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，经过标准坐标系的正交基向量的变换（投影），原坐标系上的四个点都没有发生变化，这是当然的，因为这四个点原本就是在标准坐标系下的点。现在我们将这四个点投影到另一个坐标系中。现在设定新坐标系的正交向量为：\n",
    "\n",
    "$$W = (w_{1}, w_{2}) =  \\begin{bmatrix}\n",
    "2 & 0\\\\ \n",
    "0 & 1\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "上面正交向量的表示其x轴上的“单位”向量长度是标准坐标系下基向量长度的2倍，可以预见，通过这个坐标系的变换，原有的矩形会在x轴方向上拉长2倍：\n",
    "\n",
    "$$P'=W^{T}P=(w_{1}, w_{2})^{T}(p_{1},p_{2},p_{3},p_{4})=\\begin{bmatrix}\n",
    "2 & 0\\\\ \n",
    "0 & 1\n",
    "\\end{bmatrix}\\begin{bmatrix}\n",
    "0 &  0&  1& 1\\\\ \n",
    "0 &  1&  0& 1\n",
    "\\end{bmatrix}=\\begin{bmatrix}\n",
    "0 &  0&  2& 2\\\\ \n",
    "0 &  1&  0& 1\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "直观的形象如下图所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAACkxJREFUeJzt3U+I53d9x/HXu6xizBLURifEQ/daqoaaCUmggVmTS1oQL2rw7tqeehDppXhs8SBBKIg5VZC69CBeg3v4QYLZ2NkcRQ+lSS5dqE3GdEK2mMmnh/m1v3QxO3++v93vb9/zeMDCb2Z+8/u+efObJ9/5fffH1BgjANzd/mDuAQCYTswBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdo4NydOtD9998/Lly4cKcO94Hefvvt3HvvvXOPsRHs4pA9rNjFyqbs4tq1a78ZY3zyqPvdsZhfuHAhu7u7d+pwH2ixWGRnZ2fuMTaCXRyyhxW7WNmUXVTVa8e5n5dZABoQc4AGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGxBygATEHaEDMARoQc4AGxBygATEHaEDMARqYFPOqerSqfl5VL1TVs+saCoCTmXpm/lqSL4wxnkjyqar67BpmAuCEJv0N0DHG9fd9+G6Sg2njAHAaNcaY/iBVn0vy92OMv7jp85eSXEqSra2thy9fvjz5WFPt7+/n/Pnzc4+xEezikD2s2MXKpuzi4sWL18YY20fdb3LMq+oTSX6a5Cs3nan/P9vb22N3d3fSsdZhU/7i9iawi0P2sGIXK5uyi6o6VsynXgA9l+RHSb51q5ADcHtNvQD65SSPJPlOVS2q6vE1zATACU29APrjJD9e0ywAnJI3DQE0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QAOTYl5VD1bVK1V1o6rOrWsoAE5m6pn5G0meTHJ1DbMAcEqTzqbHGDeS3KiqNY0DwGl4aYQz7Z9efj0/fPmdfP/XL809ykbY27OL//XHH/1dduYe4gRua8yr6lKSS0mytbWVxWJxOw93LPv7+xsxxyawi+SHL7+T1986SLI39ygb4eDgIHt7dvH6f72X//zouKt+Pm5rzMcYzyV5Lkm2t7fHzs7O7TzcsSwWi2zCHJvALrI8C93L83/z9NyjbATPiUNf/cFL2dvbu6t2MfV/s3yoqq4keSjJ81X16HrGAuAkpl4A/V2Sp9Y0CwCn5E1DAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNDA55lX1bFW9UFXfW8dAAJzcpJhX1eeT3DvGeCLJh6vqkfWMBcBJnJv4/Y8nubK8fSXJY0n+ZeJjwh3z8r+9kST56g9emnmSzbC3906+/2u7+OW/v5UH75l7ipOZGvOPJfnX5e3fJvmT93+xqi4luZQkW1tbWSwWEw833f7+/kbMsQnsYmVvb2/uETbCwcGBXSR58J7kT//w4K76+Zga870k9y1v37f8+P+MMZ5L8lySbG9vj52dnYmHm26xWGQT5tgEdpG8umMP72cXK3fbLqZeAH0pyZPL208luTrx8QA4hUkxH2O8kuRGVb2Q5L0xxi/WMxYAJzH1ZZaMMf56HYMAcHreNATQgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2IOUADYg7QgJgDNCDmAA2cOuZV9XRV/aqqXlznQACc3JQz86tJHlrXIACc3rnTfuMY480kqar1TQPAqXjNHKCBGmPc+g5VDyS5fNOnr48xnll+/cUxxp99wPdeSnIpSba2th6+fPnmh7nz9vf3c/78+bnH2Ah2ccgeVuxiZVN2cfHixWtjjO2j7ndkzI98gFvE/P22t7fH7u7upGOtw2KxyM7OztxjbAS7OGQPK3axsim7qKpjxXzK/2bZrqorST5TVVeq6iOnfSwApplyAXQ3yVNrnAWAU3IBFKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAbEHKABMQdoQMwBGhBzgAZOHfOq+npVXV3++9o6hwLgZKacmf9sjPFYkieSfHNN8wBwCqeO+Rjj1eXNd5McrGUaAE6lxhjTHqDqr5J8fIzxd7/na5eSXEqSra2thy9fvjzpWOuwv7+f8+fPzz3GRrCLQ/awYhcrm7KLixcvXhtjbB91vyNjXlUPJLm5wtfHGM9U1aNJ/jbJl8YYtzw7397eHru7u0fNc9stFovs7OzMPcZGsItD9rBiFyubsouqOlbMzx11hzHG9SQ7v+cAn07y3SRfPCrkANxeUy6AfjvJVpKfVNWiqu5Z00wAnNCRZ+YfZIzxjXUOAsDpedMQQANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANiDtCAmAM0IOYADYg5QANH/kHntR2o6j+SvHZHDnZr9yf5zdxDbAi7OGQPK3axsim7+KMxxiePutMdi/mmqKrd4/yl67PALg7Zw4pdrNxtu/AyC0ADYg7QwFmM+XNzD7BB7OKQPazYxcpdtYsz95o5QEdn8cwcoJ0zF/Oq+npVXV3++9rc88ytqp6uql9V1YtzzzKHqnq2ql6oqu/NPcvcqurBqnqlqm5U1bm555lLVT1aVT9fPi+enXue4zpzMU/yszHGY0meSPLNuYfZAFeTPDT3EHOoqs8nuXeM8USSD1fVI3PPNLM3kjyZw+fEWfZaki8snxefqqrPzj3QcZy5mI8xXl3efDfJwYyjbIQxxptjjP+ee46ZPJ7kyvL2lSSPzTjL7MYYN8YYb849x9zGGNfHGDeWH941nThzMX+fv0zy07mHYFYfS/LW8vZvk3x8xlnYMFX1uST3jzF+Ofcsx9H2dbGqeiDJ5Zs+fX2M8UxVPZrkz5N86c5PNo9b7WOOeTbEXpL7lrfvW34MqapPJPmHJF+Ze5bjahvzMcb1JDs3f76qPp3ku0m+OMa4K359WocP2scZ91KSbyT55yRPJfnHWadhIywv/v4oybeWPzd3hbP4Msu3k2wl+UlVLarqnrkHmlNVbVfVlSSfqaorVfWRuWe6U8YYryS5UVUvJHlvjPGLuWeaU1V9aPlceCjJ88vfYM+iLyd5JMl3lo14fO6BjsObhgAaOItn5gDtiDlAA2IO0ICYAzQg5gANiDlAA2IO0ICYAzTwP3nQg0wfom97AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "v = np.asmatrix([[0, 0], [0, 1], [1, 1], [1, 0], [0, 0]], dtype=np.float32)\n",
    "W = np.asmatrix([[2, 0], [0, 1]], dtype=np.float32)\n",
    "v1 = (W * v.T).T\n",
    "fig = plt.figure(\"test\")\n",
    "ax = fig.add_subplot(111)\n",
    "ax.grid(True)\n",
    "ax.plot(v1[:, 0], v1[:, 1])\n",
    "plt.xlim(-2.5, 2.5)\n",
    "plt.ylim(-2.5, 2.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面的例子可以总结出一些规律：变换坐标系正交向量的模决定向量变换后的模，如果正交向量的模大于1则会拉长向量，如果小于1则会缩小向量；而正交向量和原坐标系正交基向量的角度决定了向量变换后的旋转角度，由于上面两个例子中变换坐标系的正交向量与原坐标系的正交基向量是平行的，因此变换后的图形只是在大小上发生了变化而并没有产生旋转。接下来，我们可以做一些复杂的变换，现有新的变换坐标系向量：\n",
    "\n",
    "$$W = (w_{1}, w_{2}) =  \\begin{bmatrix}\n",
    "2 & 0.3\\\\ \n",
    "1 & 1\n",
    "\\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述向量组不但会在长度上拉伸原向量，也会使原向量产生一定的旋转，直观上看会使向量朝着逆时针旋转一定的角度：\n",
    "\n",
    "$$P'=W^{T}P=(w_{1}, w_{2})^{T}(p_{1},p_{2},p_{3},p_{4})=\\begin{bmatrix}\n",
    "2 & 1\\\\ \n",
    "0.3 & 1\n",
    "\\end{bmatrix}\\begin{bmatrix}\n",
    "0 &  0&  1& 1\\\\ \n",
    "0 &  1&  0& 1\n",
    "\\end{bmatrix}=\\begin{bmatrix}\n",
    "0 &  1&  2& 3\\\\ \n",
    "0 &  1&  0.3& 1.3\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "在坐标系中的表示如下图所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAEXlJREFUeJzt3WtwnOV5h/H/bcta2Tr4gA7EdsABAw7mVCywnWKQgCmEtpTAwKRpaBMChjJJOpOEUAgQSGgnpO0wKZ209ZeGpE2UxjhAIUAgiTg0nCw6xYcETMAuJhGSsY21lnW++2HXWmHZyN5d613dun4zmlmJd7W3HsTFM+9q9zV3FwBgYpuS9AAAgMIRcwAIgJgDQADEHAACIOYAEAAxB4AAiDkABEDMASAAYg4AAZSN1wPV1tb6ggULxuvhDmj37t2qrKxMeoySwFpksA45rEVOqaxFW1vbNnevG+u4cYv5ggULtHbt2vF6uANqbW1VU1NT0mOUBNYig3XIYS1ySmUtzGzLwRzHaRYACICYA0AAxBwAAiDmABAAMQeAAIg5AARAzAEgAGIOAAEQcwAIgJgDQADEHAACIOYAEAAxB4AAiDkABEDMASAAYg4AARBzAAig4Jib2RfM7JliDAMAyE9BMTezlKRTizQLACBPhe7Mr5Z0bzEGAQDkz9w9vzuaTZP0H+5+hZk94+5n7eeYlZJWSlJDQ8OSlpaWgoYthnQ6raqqqqTHKAmsRQbrkMNa5JTKWjQ3N7e5e+NYx5UV8BhXSvr++x3g7qskrZKkxsZGL4UrXZfKFbdLAWuRwTrksBY5E20tCjnNcoKkvzSzRyUtNrPPFWkmAMAhyntn7u437r2dPc1yT3FGAgAcqqL8nfn+zpcDAMYPLxoCgACIOQAEQMwBIABiDgABEHMACICYA0AAxBwAAiDmABAAMQeAAIg5AARAzAEgAGIOAAEQcwAIgJgDQADEHAACIOYAEAAxB4AAiDkABEDMASAAYg4AARBzAAiAmANAAMQcAAIg5gAQADEHgACIOQAEQMwBIABiDgABEHMACICYA0AAxBwAAiDmABAAMQeAAIg5AARAzAEggIJibmZLzeyXZva0md1drKEAAIem0J35FknnuvsKSfVmdnIRZgIAHKKyQu7s7u0jPh2QNFjYOACAfBTlnLmZnSKp1t03FuP7AQAOjbl7Yd/AbI6k+yVdsc9OXWa2UtJKSWpoaFjS0tJS0GMVQzqdVlVVVdJjlATWIoN1yGEtckplLZqbm9vcvXGs4wqKuZmVSXpQ0h3u/vz7HdvY2Ohr167N+7GKpbW1VU1NTUmPURJYiwzWIYe1yCmVtTCzg4p5oadZLpd0hqS7zKzVzJYX+P0AAHko9AnQH0j6QZFmAQDkiRcNAUAAxBwAAiDmABAAMQeAAIg5AARAzAEgAGIOAAEQcwAIgJgDQADEHAACKOjl/ABwKHr6B9XZ1auOrl51dvWoo6tXHbt61ZG93Zn9WHn2Mbp6xTFJjzuhEHMABXF3pXsHhsPcme5Vx66e4Wh3dPVkg92rd/f0j7r/FJNqpk/Tzu7MP6utSumU+bPG+8eY8Ig5gP1yd+3o7n9PjPfe7uzK7aY7dvVqT//oi4yVl01RfXVK9dUpHVtXpeXHHpH9vEJ1NSnVVaVUX5PS5m3dumnNy9rZ3a/LTp+vW//ow5o1ozyBn3hiI+bAJDMwOKRt6b5Rke7s6tXGN3p094b/VueuHnWme9U/OPp6B1WpMtVXp1RXndlB7w12fU0m1HuDXTO9TGZ2wDl29w7o7x57Rfc+u1lzZ07XvVedqXOOrzuMP3lsxBwIInc+evROumPEeep3dvdpf9ekmVNZrhk2pA/NLNPCutpsnLM76RHBnlFeeDaeerVTN61Zp9++u0d/vuxo3XDhIlWlyFEhWD2ghLm7unoHhp8k7NznCcORt7t6Bkbdf+oUU21VueqrKzR3ZoVO++BM1Q3vnlOqr8ncrq1KqbxsSvbqOksP28+zs7tPdz78K61u26pj6ir1n9cu1xkL5hy2x5tMiDlQAl7cvF0vvLF9vzvrnv6hUcenyqYMn9Y4vqFaZy2sVX1NheqqUqobsaOeU1muqVMOfKpjPD2y7ne69YEN2tHdp+ubjtXnzztOFdOmJj1WGMQcSNC73f268+GN+lHbVklSdapsOManfXDW6HPRNSnVVVeopuL9z0eXko6uHn31gQ16ZH27Fs+t0Xc+fYZOmjcz6bHCIeZAQh5d365bH1iv7bszO9XrmxeGOm/s7lrdtlV3Pvwr7ekf1JcvPEHXrDhG06byWsXDIc5vDjBBdHT16PYHN+gn69p14gdq9G+firdTfXN7t27+8To9vWmbzlgwW9+47BQdW1eV9FihEXNgnLi77nvpLX39oY3a0z+oGy44QSvPjrVTHRpyfffZzfrmY6/IJH3tTxbrk0uP1pQSOW8fGTEHxsHWHd26+cfr9dSrnVpy9GzdddkpWlgfa6f6WkeXbrxvndq27NDZx9fpbz92kubPnpH0WJMGMQcOo6Eh1/ee26K7Hv21JOmOixfrymWxdqr9g0Na9dTr+tYTmzS9fKr+4fJTdenp8ybME7RREHPgMPlNZ1o3rn5ZawPvVNe/9a6+vPplbfzdLl108pG64+KTVFedSnqsSYmYA0U2vFP92SZNnzZVf3/5qbos2E61p39Q3/rZJq166nXNqSzXv3zydF140geSHmtSI+ZAEe27U7394sWqr65IeqyienHzdt24+mW9vm23rmicr69cdKJmzpiW9FiTHjEHimDkTnX2jJg71XTvgL756K/13We3aP7s6fr3zyzVWcfVJj0Wsog5UKCRO9XLl8zXLX8Yb6f6i1c69JU16/S7XT369O8v0Jf+4ARVBnqBUwT82wDytO9O9XufOVMrjov1Fq47dvfp6w9t1Jr/eUsL66u0+rqPaMnRs5MeC/tBzIE8PPlqp27OvoXrpz6yQDdcEGun6u76ybp2ffXB9drZ3a/PnbtQnz13oVJlvDFWqYrz2weMg53dffraQxu15qW3dGxdpVZft1xLjo71Fq4du3p0y/3r9dONb+vkeTP13auW6sS5NUmPhTEQc+AguLseWd+u2x7I7FQ/25zZqUZ5C9f+wSFtS/fqyTf79bnWJ9U3MKS//ugiXX3Wh1QW6O0GIiPmwBg6dvXo1gfW67ENb+ukeTW696oztXjuxHhjrD19g++5kEXn3tt7P7IXXt7enbv60JkfmqNvXHqyjuGNsSYUYg4cgLvrR21bdedDG9VbQjtVd9eunoFMmPd7ebhMsDt39aqrd/TVh8qmmGqzF1OeP3u6fu+o2cPvlb7jzU26/tJlod5uYLIg5sB+vLm9WzetWadnXtumMxfM0TcuO/w71aEh1zu7+94T430vD9eZztzuHRh99aGKaVOGL2Kx6MhqnX1c3Yhrd2auQlRfk9KcGeUHjHXrnjcI+QRFzIERBve+heujr2iKSV+/5CT92ZlHFRS4voEhdaZ7s9fv7HnPxZVH7qa3pfs0ODT6Sss1FWXD1+pcctTsbKArslcdyt2uTk2cqw+h+AqOuZndLalR0kvu/leFjwQkY9PbXbrxvpf10v/tVNMJdfqbj52sebOmH/D47r6BA57mGHnh5R3d/aPuayYdUVk+fHHlRUdWj7o8XH11heqqU2GeZMXhVVDMzex0SZXuvsLM/tnMznD3F4s0GzAuBoZc9/xsk+75+WuakZqq2//4RH1kYa3e6Nyt519/J/fkYTr3hGFHV6/S+zkfPW2qZS+qXKGjjpihxgWzh3fO9SN20UdUlid+7h2xFLozXy7pieztJyQtk0TMMWGsbtuqL/20W9KrkqS+7iHd/l8bRx03o3zqcIw/PLdG51Tnds4jd9Kzpk/jnDMSUWjMZ0n6Tfb2u5IWj/yHZrZS0kpJamhoUGtra4EPV7h0Ol0Sc5QC1kLa8GbuFMjyuVM1KzVFs1KmmSnTrOzHzJRpetneQPdmP7K6pKEuqf23Uvu4Tn548DuRM9HWotCY75S096VhNdnPh7n7KkmrJKmxsdGbmpoKfLjCtba2qhTmKAWshdQk6ayf/0Lnnduc9Cglgd+JnIm2FoWetHtW0nnZ2+dLeq7A7weMu6mcFkEABcXc3V+S1GNmT0sacvcXijMWAOBQFPynifw5IgAkj7+NAoAAiDkABEDMASAAYg4AARBzAAiAmANAAMQcAAIg5gAQADEHgACIOQAEQMwBIABiDgABEHMACICYA0AAxBwAAiDmABAAMQeAAIg5AARAzAEgAGIOAAEQcwAIgJgDQADEHAACIOYAEAAxB4AAiDkABEDMASAAYg4AARBzAAiAmANAAMQcAAIg5gAQADEHgACIOQAEQMwBIIC8Y25m15jZc9mPTxRzKADAoSlkZ/64uy+TtELSF4s0DwAgD3nH3N03Z28OSBosyjQAgLwU45z5dZLuL8L3AQDkydz9/Q8wO1JSyz5fbnf3j5vZUkm3SLrE3Uftzs1spaSVktTQ0LCkpWXfbzP+0um0qqqqkh6jJLAWGaxDDmuRUypr0dzc3ObujWMdN2bMD3hHs3mSfijpYnffPtbxjY2Nvnbt2rweq5haW1vV1NSU9BglgbXIYB1yWIucUlkLMzuomBdymuU2SQ2S1phZq5lNL+B7AQAKUJbvHd392mIOAgDIHy8aAoAAiDkABEDMASAAYg4AARBzAAiAmANAAMQcAAIg5gAQADEHgACIOQAEQMwBIABiDgABEHMACICYA0AAxBwAAiDmABAAMQeAAIg5AARAzAEgAGIOAAEQcwAIgJgDQADEHAACIOYAEAAxB4AAiDkABEDMASAAYg4AARBzAAiAmANAAMQcAAIg5gAQADEHgACIOQAEQMwBIABiDgABFBxzM3vQzO4sxjAAgPwUFHMzO1VSRZFmAQDkqdCd+eclfbsYgwAA8mfunt8dzRZJ+gtJj0k6391v2c8xKyWtlKSGhoYlLS0tBYxaHOl0WlVVVUmPURJYiwzWIYe1yCmVtWhubm5z98axjisb6wAzO1LSvhVul7RL0m2SFh3ovu6+StIqSWpsbPSmpqaxHu6wa21tVSnMUQpYiwzWIYe1yJloazFmzN29XVLTvl83s8ckfUfSHElHmNnj7v5ksQcEAIxtzJgfiLtfIElm1qTMaRZCDgAJyTvme7l7q6TWgicBAOSNFw0BQADEHAACIOYAEAAxB4AAiDkABEDMASAAYg4AARBzAAiAmANAAMQcAAIg5gAQADEHgACIOQAEQMwBIABiDgABEHMACICYA0AA5u7j80BmnZK2jMuDvb9aSduSHqJEsBYZrEMOa5FTKmtxtLvXjXXQuMW8VJjZWndvTHqOUsBaZLAOOaxFzkRbC06zAEAAxBwAApiMMV+V9AAlhLXIYB1yWIucCbUWk+6cOQBENBl35gAQzqSOuZk9aGZ3Jj1HUszsGjN7LvvxiaTnSYKZ3W1mT5vZt5KeJUlmttTMfpldi7uTnqcUmNkXzOyZpOc4WJM25mZ2qqSKpOdI2OPuvkzSCklfTHqY8WZmp0uqdPcVksrN7IykZ0rQFknnZtei3sxOTnqgJJlZStKpSc9xKCZtzCV9XtK3kx4iSe6+OXtzQNJggqMkZbmkJ7K3n5C0LMFZEuXu7e7ek/10sv4+jHS1pHuTHuJQTMqYm9kiSR2SdiY9S4m4TtL9SQ+RgFmSdmVvvytpdoKzlAQzO0VSrbtvTHqWpJjZNEnnuPvPk57lUJQlPcDhZGZHSmrZ58vtyvwHfJukReM+VAIOtA7u/nEzWyrpIkmXjP9kidspqSZ7u0aT/H/uZjZH0j9JuiLpWRJ2paTvJz3EoZqUf5poZo9JcklzJB0h6Sp3fzLZqcafmc2T9ENJF7v79qTnGW/Zc+bXuvu1ZvZtSd9x9xeSnisJZlYm6UFJd7j780nPkyQzu0vSaco0Yqmk29z9nmSnGtukjPleZtYk6Xx3vyXpWZJgZv8q6VxJb2W/9FF335PgSOMu+1csp0v6X3f/bNLzJMXM/lTSP0rakP3STe7+bIIjlQQze8bdz0p6joMxqWMOAFFMyidAASAaYg4AARBzAAiAmANAAMQcAAIg5gAQADEHgACIOQAE8P9nt9UbDnFiXwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "v = np.asmatrix([[0, 0], [0, 1], [1, 1], [1, 0], [0, 0]], dtype=np.float32)\n",
    "W = np.asmatrix([[2, 1], [0.3, 1]], dtype=np.float32)\n",
    "v1 = (W * v.T).T\n",
    "fig = plt.figure(\"test\")\n",
    "ax = fig.add_subplot(111)\n",
    "ax.grid(True)\n",
    "ax.plot(v1[:, 0], v1[:, 1])\n",
    "plt.xlim(-5, 5)\n",
    "plt.ylim(-5, 5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，我们来做一个特殊一些的变换：将这个矩形旋转一定的角度但却不改变其形状。通过上面的例子可以知道，要达到这个目的必须使变换坐标系的两个向量正交且保证其为单位向量，这样就不会改变原来矩形的形状和大小。而因为有旋转要求，因此新的坐标系必须和原坐标系有一定的角度：也就是说新坐标系的正交基向量与原坐标系的正交基向量必须成一定的角度。根据三角函数的相关知识，我们可以非常轻松地写出这两个新坐标系的正交基向量：\n",
    "\n",
    "$$W = (w_{1}, w_{2}) =  \\begin{bmatrix}\n",
    "cos\\theta & -sin\\theta\\\\ \n",
    "sin \\theta & cos\\theta\n",
    "\\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "假如我们想让矩形旋转60度，则可以写出这个矩阵：\n",
    "\n",
    "$$W = (w_{1}, w_{2}) =  \\begin{bmatrix}\n",
    "0.5 & -0.866\\\\ \n",
    "0.866 & 0.5\n",
    "\\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "变换后的向量组为：\n",
    "\n",
    "$$P'=W^{T}P=(w_{1}, w_{2})^{T}(p_{1},p_{2},p_{3},p_{4})=\\begin{bmatrix}\n",
    "0.5 & 0.866\\\\ \n",
    "-0.866 & 0.5\n",
    "\\end{bmatrix}\\begin{bmatrix}\n",
    "0 &  0&  1& 1\\\\ \n",
    "0 &  1&  0& 1\n",
    "\\end{bmatrix}=\\begin{bmatrix}\n",
    "0 &  0.866&  0.5& 1.366\\\\ \n",
    "0 &  0.5&  -0.866& -0.366\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "变换后的图形为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAETpJREFUeJzt3X+Q1PV9x/HXG46fd4Agd4vKr0TFn8jA7XFopbkDNEOcSY0gg9gWCeUsNeqY1CYNnWSm6cQ6E2NDY2hxILnGgasViJ2m44/T3AwGKewdjWiqIY6Sop6CInLURX58+scdXbwCt7vfvf1897PPxwwze8feft/zZu/F3vu77/uac04AgNI2wHcBAIDoCHMACABhDgABIMwBIACEOQAEgDAHgAAQ5gAQAMIcAAJAmANAACqKdaCxY8e6yZMnF+twZ3XkyBFVVlb6LiMW6EU3+pBBLzLi0ov29vYDzrnqvu5XtDCfPHmyUqlUsQ53Vm1tbWpoaPBdRizQi270IYNeZMSlF2a2N5v7MWYBgAAQ5gAQAMIcAAJAmANAAAhzAAgAYQ4AASDMASAAhDkABIAwB4AAEOYAEADCHAACQJgDQAAIcwAIAGEOAAEgzAEgAIQ5AASAMAeAAEQKczOrN7NtZrbVzB4uVFEAgNxEfWW+V9Ic59xsSTVmNrUANQEAchTpGqDOuc7TPjwu6US0cgAA+TDnXPQHMbtG0gPOuZt6fb5JUpMkJRKJ2paWlsjHiqqrq0tVVVW+y4gFetGNPmTQi4y49KKxsbHdOZfs636Rw9zMxkj6maRFvV6pf0oymXSpVCrSsQohLlfcjgN60Y0+ZNCLjLj0wsyyCvOoJ0ArJD0m6f5zBTkAoH9FPQF6q6Q6SQ+aWZuZXVuAmgAAOYp6AnSjpI0FqgUAkCeWhgAgAIQ5AASAMAeAABDmABAAwhwAAkCYA0AACHMACABhDgABIMwBIACEOQAEgDAHgAAQ5gAQAMIcAAJAmANAAAhzAAgAYQ4AASDMASAAhDkABIAwB4AAEOYAEADCHAACQJgDQAAIcwAIAGEOAAEgzAEgAIQ5AASAMAeAABDmABAAwhwAAkCYA0AACHMACABhDgABIMwBIACEOQAEgDAHgABECnMzu9DMOswsbWYVhSoKAJCbqK/MP5A0V9L2AtQCAMhTpFfTzrm0pLSZFagcAEA+mJkDQADMORf9QczaJM1zzh3v9fkmSU2SlEgkaltaWiIfK6quri5VVVX5LiMW6EU3+pBBLzLi0ovGxsZ251yyr/v160lL59xaSWslKZlMuoaGhv48XFba2toUhzrigF50ow8Z9CKj1HoR9d0sg8ysVdI0SU+bWX1hygIA5CLqCdBjkuYVqBYAQJ44AQoAASDMASAAhDkABIAwB4AAEOYAEADCHAACQJgDQAAIcwAIAGEOAAEgzAEgAIQ5AASAMAeAABDmABAAwhwAAkCYA0AACHMACABhDpSBtz78WP/20ts6duKk71LQT/r1GqAA/Pn4kxN66pV39ET7Pm17/X05Jy1KjteDC66RmfkuDwVGmAMBcc4ptfegnkjt0893v6Ouo8c1Ycww3Tv3Uh1OH9e6F97QBaOG6b4bpvguFQVGmAMBeOvDj7W5fZ82dezTm+//j4YPHqgvTL1AC2vHa+bkMRowwOSc06GPj+kHz+3RuFFDddvMib7LRgER5kCJOtMYZdZnx+grcy7V/KvHqXLIp7+9zUwP3DJV+w8f1aotu1UzYojmXpHwVD0KjTAHSsi5xigLZozXhDHDz/n1gwYO0I9un6HFa7frrg0d2rhilqZPHF2k6tGfCHOgBGQzRslW5ZAKrb+jTres+aWWN6e0aeV1+szYyn6sHsVAmAMxlesYJRfVI4aoedlMLVizTUvX79CmldepesSQAlaPYiPMgRiJOkbJxWerq7TujjoteXS7ljfv1MYVswr22Cg+whyIgUKOUXIxY+Jo/fC2GWr6aUp3bejQH05y/XIc9D/CHPCkP8couZh3ZULfuflqrdrysk50VWhuo2OpqAQR5kARFXOMkovb6yfp3UNprX7+t/q71j0sFZUgwhwoAl9jlFzcd8MU7XrtTZaKShRhDvSTuIxRsmVmWnrVYA2oHM1SUQmK17MJKHFxHaNkq2KAsVRUoghzoABKYYySLZaKShNhDuSp1MYouageMUT/9OV6lopKSOk+2wAPSn2MkovPjK3UuqVJ3XbaUlEp/wcVOv5lgCyENEbJxfReS0WP/nFSgwZygbI4IsyBswh5jJKLeVcm9Dc3T9U3t+zWqi27uVJRTEV+NprZw5KSkjqcc/dGLwnwp5zGKLlYUj9RnYc+1urnf6txo4bpqywVxU6kMDezGZIqnXOzzWyNmdU553YWqDagaMp1jJKL+26YoncOpbX6uT26gKWi2In6yvxaSa09t1slzZJEmKNk/Ouv3tY9Tx2RnnpeUnmOUbJlZvruLVP1HlcqiiVzLv/fkmZmqyS1O+eeMrN5kq5zzv31aX/fJKlJkhKJRG1LS0vUeiPr6upSVVWV7zJigV5IW/cd07qXP5Ekfef3hmnCiPI+uZfNcyJ93Olvd6T1dtdJfX3mUF183sAiVVdccfn+aGxsbHfOJfu6X9SXHh9KGtlze2TPx//HObdW0lpJSiaTrqGhIeLhomtra1Mc6ogDeiE1SBr5L61avesTPfNepdbNr9PgivIN9GyfE7X1R7VgzTY9svu4Nq2cFeRSUal9f0R91r4oaW7P7XmStkd8PKDoplVX6IEvTdXWPQf0jc0vKcpPq+WiesQQNX95piRp6fod2n/4qOeKECnMnXMdktJmtlXSSefcjsKUBRTXoroJum/eFG3ueEvfe+Y13+WUhFNLRe8dTmt5804dOXrcd0llLfLPk865e51zs51zXylEQYAv98y9RLfNnKBHfvG6frp9r+9ySsKppaKX3zqkuzZ06NiJk75LKlvlOxwEejEzfecPrtbcy2v07Sdf1tOvdPouqSScWipqe22/Vm3ZzZjKE8IcOE3FwAH6+yXTNXX8ebpn4y617/3Ad0klYUn9RN0z5xI9ntqnh1v3+C6nLBHmQC/DB1do/dKkLhg1VMubU3p9f5fvkkrCfTdM0a2147X6uT3auON3vsspO4Q5cAbnV3W/W6NigGnp+h1673Dad0mxd2qp6HNTqrVqy24991/v+i6prBDmwFlMOr9S6++o0wdHPtGyH+9UF+/W6NOggQP0o9tn6KoLR+muDR3a9buDvksqG4Q5cA7XjD9Pj9w+Q692HtbKx9r1yXHerdGXU1cqqhnRPaZ648AR3yWVBcIc6EPjZTUsFeWIpaLiI8yBLCyqm6Cv3sBSUS5YKiouwhzI0t1zWCrK1fSJo/XIEpaKioEwB7LEUlF+5l7BUlExEOZADlgqys/iugm68cqEHk/tU/O2N32XEyTCHMgRS0XZe+PAEX3v6dd0/YPP65lfv6uRQyt03vDBvssKEpdSAfJwaqlowZptWrp+hzb/2XWqGTHUd1mxcDh9TD9/qftC2Km9BzXApN+fUq1v3nSF5l2R0NBBYV7MwjfCHMjTqaWixWu3a9mPd+qf77xWVWV6qbmTJ522vf6+nmj/bz31SqfSx07q4upKfWP+5frS9IuUGMl/dP2tPJ95QIGcWir6k+aUVj7WrvV31GnQwPKZXr5x4Ig2te/T5o59evtQWiOHVmhh7XgtrJ2gaeNHyYwLYRcLYQ5E1HhZjR64Zar+4omX9PVNL+mhW6cFHWKMUeKJMAcKYFFygjoPpfX9Z3+jC0YN1f2fv9x3SQV14qTTi73GKJfUVDFGiRHCHCiQu+dconcOpfXIL17XuFHD9EezJvkuKTLGKKWDMAcKpHup6CrtP5zWt598WTUjhujzV43zXVbOGKOUJsIcKKCKgQO0+rbpWvLof+iejbu0YUW9aieN8V1Wn06NUf7xV2nteq6VMUoJIsyBAhs+uELrlia1YM02LW9OadPK63RxdZXvss6o9xhleIW0MDmRMUoJIsyBfhDnpaJzjVEG739NN86d6rtE5IEwB/pJnJaKsn03Slvbb7zUh+gIc6Af+V4q4t0o5YMwB/pZsZeKeDdKeSLMgSLo76UilnpAmANF0h9LRYxRcAphDhRJ76WixIghujGPpaKP0sf074xR0AthDhTR6UtFd2/cpQ0rZql20ug+v44xCvpCmANFdmqpaOE/vKjlzTvPuVR0pjHKrbUTtLB2vK5hjILTEOaAB+dXDVHzspm6Zc0v/99S0ZnGKJ+bUq1VN12puVfUMEbBGRHmgCcTzx/+qaWiP7/xMj35n299aozyl/Mv182MUZAFwhzw6PSlomU/2ckYBXkjzAHPGi+rUfOymfoofUxzLmeMgvwQ5kAMXH/pWN8loMTl/UsizGy+mb1qZi8UsiAAQO6i/Maf7ZKmFaoQAED+8h6zOOcOSuIEDQDEQPF+FycAoN+Yc+7cdzAbJ6ml16c7nXOLe/7+Befc9Wf52iZJTZKUSCRqW1p6P0zxdXV1qaoqnpfwKjZ60Y0+ZNCLjLj0orGxsd05l+zrfn2GeZ8PcI4wP10ymXSpVCrSsQqhra1NDQ0NvsuIBXrRjT5k0IuMuPTCzLIK8yjvZkmaWaukq82s1cxYUQMAT6KcAE1JmlfAWgAAeeIEKAAEgDAHgAAQ5gAQAMIcAAJAmANAAAhzAAgAYQ4AASDMASAAhDkABIAwB4AAEOYAEADCHAACQJgDQAAIcwAIAGEOAAEgzAEgAIQ5AASAMAeAABDmABAAwhwAAkCYA0AACHMACABhDgABIMwBIACEOQAEgDAHgAAQ5gAQAMIcAAJAmANAAAhzAAgAYQ4AASDMASAAhDkABIAwB4AAEOYAEADCHAACkHeYm9kKM9ve82dJIYsCAOQmyivzZ51zsyTNlvS1AtUDAMhD3mHunHuz5+ZxSScKUg0AIC/mnIv2AGYrJY12zn33DH/XJKlJkhKJRG1LS0ukYxVCV1eXqqqqfJcRC/SiG33IoBcZcelFY2Nju3Mu2df9+gxzMxsnqXcKdzrnFptZvaS/knSzc+6cr86TyaRLpVJ91dPv2tra1NDQ4LuMWKAX3ehDBr3IiEsvzCyrMK/o6w7OuU5JDWc4wEWSHpL0xb6CHADQv6KcAP2WpISkzWbWZmbDClQTACBHfb4yPxvn3J2FLAQAkD+WhgAgAIQ5AASAMAeAABDmABAAwhwAAkCYA0AACHMACABhDgABIMwBIACEOQAEgDAHgAAQ5gAQAMIcAAJAmANAAAhzAAgAYQ4AASDMASAAfV7QuWAHMtsvaW9RDnZuYyUd8F1ETNCLbvQhg15kxKUXk5xz1X3dqWhhHhdmlsrmStflgF50ow8Z9CKj1HrBmAUAAkCYA0AAyjHM1/ouIEboRTf6kEEvMkqqF2U3MweAEJXjK3MACE7ZhbmZrTCz7T1/lviuxzczm29mr5rZC75r8cHMHjazrWb2A9+1+GZmF5pZh5mlzazCdz2+mFm9mW3reV487LuebJVdmEt61jk3S9JsSV/zXUwMbJc0zXcRPpjZDEmVzrnZkgabWZ3vmjz7QNJcdT8nytleSXN6nhc1ZjbVd0HZKLswd8692XPzuKQTHkuJBefcQefcUd91eHKtpNae262SZnmsxTvnXNo5d9B3Hb455zqdc+meD0smJ8ouzE/zp5J+5rsIeHWepI96bh+SNNpjLYgZM7tG0ljn3K9915KNYOdiZjZOUkuvT3c65xabWb2kL0i6ufiV+XGufvioJyY+lDSy5/bIno8BmdkYST+UtMh3LdkKNsydc52SGnp/3swukvSQpC8650rix6dCOFs/ytyLku6U9LikeZJ+4rUaxELPyd/HJN3f831TEspxzPItSQlJm82szcyG+S7IJzNLmlmrpKvNrNXMhvquqViccx2S0ma2VdJJ59wO3zX5ZGaDep4L0yQ93fMTbDm6VVKdpAd7MuJa3wVlg6UhAAhAOb4yB4DgEOYAEADCHAACQJgDQAAIcwAIAGEOAAEgzAEgAIQ5AATgfwHhQ7O+h9e7GAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "v = np.asmatrix([[0, 0], [0, 1], [1, 1], [1, 0], [0, 0]], dtype=np.float32)\n",
    "W = np.asmatrix([[0.5, 0.866], [-0.866, 0.5]], dtype=np.float32)\n",
    "v1 = (W * v.T).T\n",
    "fig = plt.figure(\"test\")\n",
    "ax = fig.add_subplot(111)\n",
    "ax.grid(True)\n",
    "ax.plot(v1[:, 0], v1[:, 1])\n",
    "plt.xlim(-2.5, 2.5)\n",
    "plt.ylim(-2.5, 2.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 矩阵的特征值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "矩阵的奇异值分解和方阵的特征值分解非常相似，因此在介绍奇异值分解之前先说一下方阵的特征值及其特征矩阵。现有n阶方阵A，若该方阵满足如下条件：\n",
    "\n",
    "$$Aw_{1}=\\lambda_{1}w_{1}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们就说$\\lambda_{1}$和$x_{1}$分别是方阵A的特征值和特征向量。一个方阵通常会有m个特征值和特征向量，可以写成如下形式：\n",
    "\n",
    "$$\\begin{matrix}\n",
    "Aw_{1}=\\lambda_{1}w_{1}\\\\\n",
    "Aw_{2}=\\lambda_{1}w_{2}\\\\\n",
    "...\\\\\n",
    "Aw_{m}=\\lambda_{1}w_{m}\n",
    "\\end{matrix} \\rightarrow A[w_{1},w_{2},...,w_{m}]=[w_{1},w_{2},...,w_{m}]\\begin{bmatrix}\\lambda_{1} & ... & 0\\\\\n",
    "0&\\lambda_{i}&0\\\\0&...&\\lambda_{m}\\end{bmatrix} \\rightarrow\n",
    "AU=U\\Lambda \\rightarrow\n",
    "A=U\\Lambda U^{-1}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从特征值和特征矩阵的解析表达式上可以看出，特征值和特征矩阵的几何意义在于：m维空间中存在这样一组向量，这组向量经过一个m维坐标系变换后得到的向量组中的向量和原向量组对应向量同方向但长度相差一个系数，该系数就是其对应的特征值。其几何意义如下图所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xt0VNXB/vHvJgSCBCJGiGIUkKSIxgiEu4AJUC3IRVGxglKlGhUpRa3aVsVLtdKfr7UoS2qhryIq0ar9VWmRFprUS7kFpUUuClrRqFgICAQScmG/f+zEEAi5zCVn5uT5rDUrYc7MmWcCPDmzZ589xlqLiIhEtxZeBxARkeCpzEVEfEBlLiLiAypzEREfUJmLiPiAylxExAdU5iIiPqAyFxHxAZW5iIgPtGyqBzr55JNt165dw7b/AwcO0LZt27DtP9yU3zvRnB2U32vhzr9u3bpd1tqO9d2uycq8a9eu5Ofnh23/eXl5ZGZmhm3/4ab83onm7KD8Xgt3fmPM9obcTsMsIiI+oDIXEfEBlbmIiA802Zh5bcrKyigoKKCkpCTofSUkJLB58+YQpApeXFwcycnJxMbGeh1FRJoJT8u8oKCAdu3a0bVrV4wxQe1r//79tGvXLkTJAmetpbCwkIKCArp16+Z1HBFpJjwdZikpKSExMTHoIo8kxhgSExND8mpDRKShPB8z91ORV/HjcxKRyOZ5mYuISPBU5rX43ve+x4knnsiYMWNqXD937lxSUlIwxrBr1y6P0omIHEtlXos77riDRYsWHXP9+eefz/Lly+nSpYsHqUREjq9Zl/natWtJT0+npKSEAwcOcM455/DBBx8wYsSIWmfG9O7dm3CuLyMiEqigpiYaY9KA3wEVwDZgqrXWBrKvmTNnsn79+oCzVFRUEBMTU+O6Xr168Zvf/Oa49+nXrx/jxo3jnnvuobi4mKuvvpq0tLSAM4iIeCXYeeYfWmsHAxhjngH6AmuDTtWEZs2aRb9+/YiLi+OJJ57wOo6ISECCKnNrbdkRfzwEfB7ovuo6gm6IQE8a2r17N0VFRZSVlVFSUhLVS3GKSPMV9BmgxphxwC+Bj4DCo7ZlA9kASUlJ5OXl1bhvQkIC+/fvDzYC4IZZAtnX1KlT+fnPf8727du59dZbeeyxxwA4ePAg5eXlte7TWktRURGtW7c+7n5LSkqOeb51KSoqatTtI00054/m7KD8XouY/NbakFyAJ4FLj7c9IyPDHm3Tpk3HXBeoffv2Nfo+CxcutJdeeqm11try8nLbv39/u2LFCjtkyBB78skn27i4OHvaaafZN99801pr7Zw5c+xpp51mY2Ji7Kmnnmp/+MMfHnffjX1uubm5jc4fSaI5fzRnt1b5vRbu/EC+bUAHB/sGaGtr7aHKP+4DioP71dK0pkyZwpQpUwCIiYlh9erVAAwfPrzW28+YMYMZM2Y0WT4RkYYKdpjle8aY2yq/3wr8Ncj9iYhIAIJ9A/RPwJ9ClEVERALUrE8aEhHxC5W5iIgPqMxFRHxAZS4i4gMq81ocbwncyZMn06NHD9LS0pg6dSplZWXH2YOISNNSmdfieEvgTp48mS1btrBhwwaKi4tZsGCBB+lERI7VrMu8sUvgjh49GmMMxhj69+9PQUGBB6lFRI4V9NosoRJNS+CWlZWxaNEi5syZE3BeEZFQipgy90ogS+BOmzaNYcOGMXTo0DCnExFpmIgp82hZAveBBx5g586dPP3004FGFREJuWY9Zg6QnZ3NL37xCyZPnsxdd91V520XLFjAsmXLWLx4MS1aNPsfnYhEkIg5MvfCc889R8uWLZk0aRIVFRUMHjyYv//979x3331s2bKFoqIikpOT+f3vf89FF13ETTfdRJcuXRg0aBAAEyZMYNasWR4/CxGRZl7mjV0Ct7y8vMmyiYg0hsYKRER8QGUuIuIDKnMRER9QmYuI+IDKXETEB1TmIiI+oDI/yvr16xk0aBDnnHMO6enpvPTSS99umzt3LikpKRhj2LVrl4cpRURqatbzzGtzwgkn8Nxzz5GamsqXX35JRkYGF110ESeeeCLnn38+Y8aMITMz0+uYIiI1NOsj89qWwC0tLSU1NRWAzp0706lTJ3bu3AlA79696dq1q4eJRURqF1lH5rUd8U6cCNOmwcGDMHr0sduvvRauvRZTWAhjx9bclpdX58PVtwTumjVrKC0tpXv37o1+KiIiTSmoMjfGDAAeByqAfGvtrSFJ1YSOtwTuV199xTXXXMPChQu1qJaIRLxgj8y3A8OttSXGmBeMMedaazcEvLe6jqRPOKHO7TYxsd4j8drUtgTuvn37uPjii3nooYcYOHBgo/cpItLUgjrktNbusNaWVP6xHHeEHlWOXgK3tLSUSy+9lClTpnDFFVd4HU9EpEGMtTb4nRiTDjxirb34qOuzgWyApKSkjJycnBr3S0hIICUlJejHh9o/Nq4+L774In/+85954YUXqKioYOTIkdxwww1Mnz6dnj17fnu7efPmkZ6ezrx585gzZw5ff/01HTt25MILL2Tu3Lm17nvbtm3s3bu3wVmKioqIj49vVP5IEs35ozk7KL/Xwp0/KytrnbW2b703tNYGdQFOAt4CTqnrdhkZGfZomzZtOua6QO3bty9k+wqFxj633Nzc8ARpItGcP5qzW6v8Xgt3ftz7kfV2cVDDLMaYlsDzwB3W2h3B7EtERAIX7DSNK4B+wK+MMXnGmEEhyCQiIo0U1GwWa+1iYHGIsoiISIA0gVpExAdU5iIiPqAyFxHxAZX5UepaAnfy5Mn06NGDtLQ0pk6dSllZmYdJRUSqqcyPUrUE7saNG3nzzTeZOXMm33zzDeDKfMuWLWzYsIHi4mIWLFjgcVoREadZl3ljl8AdPXo0xhiMMfTv35+CggIv44uIfEtL4AawBG5ZWRmLFi1izpw5DXteIiJhFlll7oFAlsCdNm0aw4YNY+jQoU0dV0SkVpFV5lGwBO4DDzzAzp07efrppxv9WCIi4dKsx8yhcUvgLliwgGXLlrF48WJ9YIWIRJTIOjJvYs899xwtW7Zk0qRJVFRUMHjwYHJycnjrrbcoLCzk2WefBeDZZ5+lV69e3HTTTXTp0oVBg9wSNBMmTGDWrFkePgMREadZl/mUKVOYMmUKADExMaxevfrb62tTXl7eZNlERBpDYwUiIj6gMhcR8QGVuYiID6jMRUR8QGUuIuIDKnMRER9QmR/H3Xffzemnn058fLzXUURE6qUyP46xY8eyZs0ar2OIiDSIyhx4/vnn6d+/P7169eLGG2+koqKCgQMHcuqpp3odTUSkQSLmDNCZM2H9+sDvX1HRhpiYmtf16gW/+U3d99u8eTMvvfQS7777LrGxsUybNo0XXnjhuGeBiohEoogpc6+sWLGCdevW0a9fPwCKi4vp1KmTx6lERBonqDI3xnQGlgBnA/HW2oAXL6nvCLo++/cX065du0bfz1rLD37wAx555JHgAoiIeCjYMfPdwAhgVQiyeGLEiBG88sor/Pe//wXc+ubbt2/3OJX4ibVeJ5DmIKgyt9aWWGv3hCqMF84++2weeughLrzwQtLT0/nud7/LV199xZ133klycjIHDx4kOTmZ+++/3+uoEoXeew8GDYJPP/U6iYRKSQmsXOlGE666CubP7+Z1JACMDcFhgzEmDxh59DCLMSYbyAZISkrKyMnJqXG/hIQEUlJSgn58gIqKCmKOfgfUQ9u2bWPv3r0Nvn1RUVFUz2mP5vzhyr5/f0uyszOoqDDMn7+OhISykD8GRPfPHiI7v7Xw5Zdt2LSpHZs3t2fz5vZs2xZPebk7Du7UqYTzz/+CGTM+D1uGrKysddbavg0Ia4O+AHlAy7puk5GRYY+2adOmY64L1L59+0K2r1Bo7HPLzc0NT5Am0pT59+3bZ/fv3x+y/YUje0WFtWPGWBsba+3KlSHffQ36txM6e/ZYu2yZtQ8+aO3o0dYmJlrrKt3atm2tzcy09q67rH3tNWu/+MLdJ9z5gXzbgB5u9rNZJLoUFxfTp08fMjMzmT9/vtdxjutXv4IlS+DJJ+Goj5GVCFFeDhs2wOrVsGqV+7pli9tmDJx9NowfDwMGuL/Ds8+GlhHcmMHOZokFlgLnAcuMMT+31q4OSTKRWrRp04bLL7+c2bNnM27cOMaOHet1pGPk5sI998D3vw+33OJ1GqlSUFCzuNetg4MH3baOHV1hX321+9q3LyQkeJu3sYIqc2ttGTAyRFlEGuSBBx5g6dKlXH/99XzwwQd07NjR60jf+uILV+I9esD8+e4IT5regQOurKuKe/Vq93cD0KoV9OkDN9zginvAAOjaNfr/riL4RYNI7Vq1asXzzz9PRkYG2dnZvPbaa5gI+J9YVgZXXumKJC8PIvQ9Pd85fBg+/LDmUfeGDVBR4bZ37w4XXFA9XHLeedC6tbeZw0FlLlEpLS2Nhx9+mDvuuIOFCxdy7bXXeh2Jn/4U3n0XFi+Gnj29TuNfu3bVLO41a6Bq4lj79q60f/YzV9z9+7shlOZAZV6LgwcPcsUVV/Dxxx8TExPD2LFjmT17ttex5Ci33norS5YsYcaMGWRmZtK1a1fPsrz6Kvz61zB9uhtmkdA4dAj+9a+awyUff+y2tWgB6enu5101XNKjh7u+OVKZH8dPfvITsrKyKC0tZcSIESxdupRRo0Z5HUuOEBMTw8KFCzn33HO57aqreOWtt2gRG9vkOT76CK67zpXJY481+cP7hrXu5Kojj7rfew9KS932zp1daWdnu68ZGdC2raeRayovd2cUxcfDnj3ut/svftFkD68yxy2B+8QTT1BaWsqAAQN46qmnyMrKAtz4bJ8+fSgoKPA4pdSmS5cu/O+99zLyzjt5e8IELnjjjSZ9/AMH4LLL3JtqL7/svkrD7NsHa9fC4sVn8OtfuwLfudNta9PGzSiZMaP6qDs52du8NRw+DL/9LWzdyrkrV8Lu3fCf/7jAjz0GcXFu+333Ndl8xogp80hdAvebb77hjTfe4Mc//nHg4SSsLrv9dlY//jgDlyzho1df5TuXXdYkj2st3HwzbNwIb74JZ5zRJA8blSoq3M+p6oh71SrYvLlq3ZozOessGD26urjT0sCDF1k1/fOfsGmTe+m1dau7ZGTAwoVuLOf+++HAAVqfeir07g2XXw4jRrj7tmkD//1vk06RiZgy90pdS+CWl5dz1VVXMWPGDM4880wvY0odTIsWpKxYwZu9ezPv3nv505gxtG6C6Qq/+x0sWuT+T194YdgfLqp89VXN4s7Pd69iABITXWFfeaUr70OH3mHs2CFNH3L3bjcNZuvW6sI+4QR45hm3ffp0eP9993Kre3dITXVzGqts2gSJieT/4x9kZmYeu/8mnmEVMWUeiUvgZmdnk5qaysyZM4MLJ2F3cs+etPjDH1g2bhz3339/2Jc0zs93r6gvugjuvTesDxXxiovdnO6qNyhXrYLPK5cqiY11r5Cvu676qLt795o9l5cX8MrZ9du7t/qoeutWN47z5JNu23XXweuvu+9jYtxk8wEDqu/77LPQrp17yVXbuk8nnxy+3AGImDL3yogRIxg/fjy33nornTp1Yvfu3ezfv5/58+ezd+9eFixY4HVEaaCxY8cyc9Ikhs6ezb/POIP0m28Oy+Ps3u1eUSclwfPPN6/ZE9a6TjzyqPvf/3bv/YHrw8GDq4u7d283fBxWBw/Ctm3VhX377e63yJ13wqOP1rxtly7ujcnYWHe7G26A73zHBT/6DY/09DAHD61mX+ZHLoF7+PBhYmNjmTNnDg8//DBnnXUWfSpfVk2fPp3rr7/e47RSnwcffZQ9L7+MnTGD/ePH065z55Du//BhuOYa+PJLeOediDs4C7nCQjePu6q416xxEzXAHbT26+c6c8AAd0lKClOQ0lL45BM3HHLBBe5c+xdfhLvucufpH2niRDjzTDf2lZjoyjo11b0kaNOm+nbDhoUprDeafZkDXHnllVx55ZU1rrP6RIGo1K5zZ/4zZw5pt9zCO8OHM6xq5aQQeeQR+MtfYO5cd0KKn5SWuqPsI6cGbt3qthnj3pS87LLqo+6ePWsffQhYeTls3+4K+MQT3VjWPfe4At++3f0mBfj73yEry81VHD7cFXVqqivtlBT3WwZg5Eh3aSZU5uI76dOmkffcc2SuXs2ae++lf4jm+i5fDrNmuQ8kmDYtJLv0jLXw2WfHzukuKXHbTznFFfbUqe5r377VHRmUw4fdugetW7t3Sf/nf0irmtr3ySdu28KFMGWK+02xc6cLcPXV1YWdlub2lZnpLgKozMWnBv31r3yYlESr2bPZ+aMf0THID+kuKHAl3qOHm8USAUvBNMr+/e5A98jy3rHDbYuLc5M0pk2rHi4544wQPMdDh+CFF2pO7du2zR1t//zn7jZPPUVc587uXdJLLnFlPXSo29a7t3tnVRrE8zK31kbEIkmhpCEa77Vu3x5efZWLLrmE82+6iVdffTXgf2elpW4YtqQEXnst8hfQqqhwc7iPLO6NG6tHKVJT3ehD1XBJenoQJzutWVM9va9qit/w4e6Nx5gYuOkmd7szz3QP/N3vus/RA3f4f+AA+W+9VfvUPmkUT8s8Li6OwsJCEhMTfVPo1loKCwuJC/tb+FKfHqNH85OHH+buO+9kyYMPMva++wLaz513us98fOklOOusEIcMgd27Y3n99eriXrvWHYkDdOjgxvYnTHDF3b+/G5JusP37a07t++gj9y5n1SyRiRPdeHaLFm5GSGqq+wruzMdt29zYdm1nQRoTfS9xIpinZZ6cnExBQQE7q87hDUJJSUnEFGhcXBzJEXXucfN12223kfr441xw//0UjBhB8pDGnZzy8sswZ46bUz5xYphCNkJJiTuP5ciFpz799HzA9WV6upttUzVckpragKmTxcVu9aqqsi4udmdCAYwa5ZaCrJKcXH2WI7glIjt0gG7dal9XVqfFNhlPyzw2NpZu3ULzydZ5eXn07t07JPsS/4iJiaHvyy/D0KHsGjOGzrt20aKBa2Vs2QI//KEbFTh6unJTsNZ17JHDJevXu/cIAU4/3RX2qFHbmDQphT593AmMtSorc2uHfPSRW81q+nR3/S23wLx5VefVO926uTVFjHFryZaUuN8KKSnHPkDVkIl4zvMxc5FwSx4yhLevu46hzzxD3qWXktmAxbgOHHAnBsXFNd0CWt9844agjzzqLix029q2dTNKbrut+qi7agp9Xl4BQ4akuMHy/3zmjrCHDXPhf/97N5/y00+rP60BYNIkOOkkd5SdlFRzat+Rn5d28cXhf+ISEipzaRaGLFjAqqVLGbRkCVv/+EdSL730uLe1Fm680S29sWxZeFbrq/ow4SPPpPzwQ7fNGDeHu+rDhAcMgHPOccMo9vBhvlq3jq3/2sWSJZ/TYeNGsvLz3Zk8H39cvV7s+vXuI3USE93iUN//fvXJM6mpbmgE3GD6hAmhf4LS5FTm0iyYFi3ovmIFq3v14omf/pQXRo8+7mJcv/2tm1H34INu8kUoFBTUPOLOz3dD01D9YcJVY919Mw5TWvI1Wz/5hC9WrmTPza+x9rPP6FBYyOklJXQGbgD+Akw86SRGt2/vpvaNHVtd1ikpbueXXOIu4nsqc2k2Op59NitfeYVXx48n9TiLca1Z45ZjHjUK7r47sMc5cKB6TnfVUfeXX7ptrVq56dPZ2ZCWVkTH+I0k/mMhZZs20mr+djr8cifm4EFmAU8D5wDvAwWxsezs0IH8nj0x3/kOPxszhnnDhpGcnMxbmtonqMylmRk3bhzTrrmGlFoW4yoshCuucNOfFy1q2AJaVR8mfORwyQcfVA9Pd+t6mD7dPmdi6jpSD+ZyVuEKOm78nD9uOMwNBw+SAHwDHAa+aNmSrxMSWN+9OyMyMxk/ahSp3btjTj+dbm3aEJqpAuJXKnNpdmbPns2exYu/XYwLXClffbU7K/Kdd44/F3vnzppH3GvXVn+YcNvWJfRst4nJJ66iXcv3+INdxX8+3ci/PoWqM+G/atGCHe3b0y0tjUfHjyc1NZWPW7YkecgQTk9I4PSwP3vxq6DL3BjzONAXeM9aq4/jkYjXrnNnPnn8cc790Y94Z8QImDePhx5ynxb01FNuJUBwZ6OvX++Ke+W75fzz7VI++8pNzTNU0Db+U2AVD7Kcy1lFj0Mf0uKQZacxvNupEwdGjyI1dRLri4vpdO65JGdmcmqnTpwKaBKthFpQZW6M6QO0tdYONcbMM8b0s9auDVE2kbA5b/p08hYtInPNGu55eC2/XJHJuLEVlO/YyqTz9/P+Rx3YuqcrFRXuv0hnvmYgq7iF1QxkFW3N+9zcsyepqamcfegQu5OGs6X/TzktM5OOXbpwCaC3HaUpBXtkPghYXvn9cmAgoDKXqDDob3/jtyddw/9b/mNaUM7rb7Tk9TfOog0HySCfT1q+QXKXL8hO2Mzg+G9o2bMnCf36ceqw60js0YM1zelTKSTimWAWhTLG3A2ss9a+aYwZCQy21j54xPZsIBsgKSkpIycnJ9i8x1VUVER8pK+AVAfl98ZTsz7j1bevYlDLpZyb+D7duu7gjN6xxPdOo21KCiYKCjtaf/ZVlL9uWVlZ66y1feu9obU24AtwCzCx8vsJwIzj3TYjI8OGU25ublj3H27K752lS5d7HSEo0fyzt1b56wPk2wb0cbCHHSuBqlV3RgKrgtyfSJOLiwvlx+WIeCOoMrfWvgeUGGPeBg5ba9eEJpaIiDRG0FMTraYjioh4LvLf3RERkXqpzEVEfEBlLiLiAypzEREfUJmLiPiAylxExAdU5iIiPqAyFxHxAZW5iIgPqMxFRHxAZS4i4gMqcxERH1CZi4j4gMpcRMQHVOYiIj6gMhcR8QGVuYiID6jMRUR8QGUuIuIDKnMRER9QmYuI+IDKXETEB1TmIiI+EHCZG2NGGWO2GGPeCWUgERFpvGCOzFcB54UqiIiIBK5loHe01u4BMMaELo2IiATEWGuD24Ex71hrhxxnWzaQDZCUlJSRk5MT1GPVpaioiPj4+LDtP9yU3zvRnB2U32vhzp+VlbXOWtu33htaa+u8AKcAeUddco7Y/k59+7DWkpGRYcMpNzc3rPsPN+X3TjRnt1b5vRbu/EC+bUDH1jvMYq3dAWQG9CtFRESaRDCzWfoaY5YDacaY5caYuBDmEhGRRgjmDdB8YGQIs4iISIB00pCIiA+ozEVEfEBlLiLiAypzEREfUJmLiPiAylxExAdU5iIiPqAyFxHxAZW5iIgPqMxFRHxAZS4i4gMqcxERH1CZi4j4gMpcRMQHVOYiIj6gMhcR8QGVuYiID6jMRUR8QGUuIuIDKnMRER9QmYuI+IDKXETEB1TmIiI+EHCZG2NuMMasqrxMCmUoERFpnGCOzP9mrR0IDAVuD1EeEREJQMBlbq39tPLbcqAiJGlERCQgxlob3A6MuRnoYK39ZS3bsoFsgKSkpIycnJygHqsuRUVFxMfHh23/4ab83onm7KD8Xgt3/qysrHXW2r713tBaW+cFOAXIO+qSU7ltAPAGEFPffjIyMmw45ebmhnX/4ab83onm7NYqv9fCnR/It/X0q7WWlg0o+x1A5tHXG2NOAx4DxllrNcwiIuKhYN4AnQUkAa8ZY/KMMW1ClElERBqp3iPz47HW3hjKICIiEjidNCQi4gMqcxERH1CZi4j4gMpcRMQHVOYiIj6gMhcR8QGVuYiID6jMRUR8QGUuIuIDKnMRER9QmYuI+IDKXETEB1TmIiI+oDIXEfEBlbmIiA+ozEVEfEBlLiLiAypzEREfUJmLiPiAylxExAdU5iIiPqAyFxHxAZW5iIgPBFzmxpgfGGPeMsasMcZMC2UoERFpnGCOzF+01g4DBgE3hiiPiIgEIOAyt9aWVX7bCtgcmjgiIhIIY60N/M7GzAKygTnW2kdr2Z5duZ2kpKSMnJycgB+rPkVFRcTHx4dt/+Gm/N6J5uyg/F4Ld/6srKx11tq+9d7QWlvnBTgFyDvqknPE9tbAGiCxrv1kZGTYcMrNzQ3r/sNN+b0TzdmtVX6vhTs/kG/r6WlrLS0bUPY7gMyjrzfGtLbWHgJKgYPAofp/x4iISDjUW+Z1+JkxJhM3Zr7IWlsUmkgiItJYAZe5tfb+EOYQEZEg6KQhEREfUJmLiPiAylxExAdU5iIiPqAyFxHxAZW5iIgPqMxFRHxAZS4i4gMqcxERH1CZi4j4gMpcRMQHglrPvFEPZMxOYHsYH+JkYFcY9x9uyu+daM4Oyu+1cOfvYq3tWN+NmqzMw80Yk28bsoB7hFJ+70RzdlB+r0VKfg2ziIj4gMpcRMQH/FTmv/M6QJCU3zvRnB2U32sRkd83Y+YiIs2Zn47MRUSaLV+VuTHmB8aYt4wxa4wx07zO01jGmBuMMasqL5O8ztNYxphRxpgtxph3vM7SUMaYx40xbxtj5nidJRDGmM7GmPeMMSXGmGA+07fJGWMGGGP+Wfnzf9zrPI1ljEk7Iv8zxhjjZR5flTnworV2GDAIuNHrMAH4m7V2IDAUuN3rMAFYBZzndYiGMsb0Adpaa4cCrYwx/bzOFIDdwAjczz7abAeGV/78OxljzvU6UCN9aK0dXJkfwNPpib4qc2ttWeW3rYDNXmYJhLX208pvy4EKD6MExFq7x1p7yOscjTAIWF75/XJgoIdZAmKtLbHW7vE6RyCstTustSWVf4y6f/NH9A3AIeBzr7KAz8ocwBgzC9gKrPM6SxBuAv6/1yGagROBfZXf7wU6eJil2TLGpAMnW2s3eZ2lsYwx44wxHwCdgEIvs0TVGFsVY8wpQM5RV++w1n7fWvugMeZXwNvGmP+11nr6A65NXfmNMQOA0cAlTZ+sYerK70WeIHwDtK/8vn3ln6UJGWNOAuYCE73OEghr7evA68aYJ4ExwB+9yhKVZW6t3QFkHn29MaZ15cv8UuAg7qVPxKkj/2nAY8A4a23EvuQ8Xv4otBL33srLwEjgWU/TNDOVb9g+D9xR+W8qqhzRN+Be4RV7mcdvwyw/M8bkAe8CL1lrizzO01izgCTgNWNMnjGmjdeBGsMY09cYsxxIM8YsN8bEeZ2pLtZyEjzOAAAAcklEQVTa94ASY8zbwGFr7RqvMzWWMSa28md+HrCs8pVdtLgC6Af8qvLf+yCvAzXS94wx/zDG/AP3//avXobRSUMiIj7gtyNzEZFmSWUuIuIDKnMRER9QmYuI+IDKXETEB1TmIiI+oDIXEfEBlbmIiA/8H19vEah52RMsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "A = np.asmatrix([[3, 1], [0.5, 1]], dtype=np.float32)\n",
    "eigvalue, eigvector = np.linalg.eig(A)\n",
    "v1 = A * eigvector\n",
    "fig = plt.figure(\"test\")\n",
    "ax = fig.add_subplot(111)\n",
    "ax.grid(True)\n",
    "ax.plot([0, eigvector[0, 0]], [0, eigvector[1, 0]], color=\"black\", label=\"x11\")\n",
    "ax.plot([0, eigvector[0, 1]], [0, eigvector[1, 1]], color=\"black\", label=\"x12\")\n",
    "ax.plot([0, v1[0, 0]], [0, v1[1, 0]], color=\"red\", linestyle=\"--\", label=\"x21\")\n",
    "ax.plot([0, v1[0, 1]], [0, v1[1, 1]], color=\"red\", linestyle=\"--\", label=\"x22\")\n",
    "ax.plot([0, A[0, 0]], [0, A[0, 1]], color=\"blue\", label=\"e1\")\n",
    "ax.plot([0, A[1, 0]], [0, A[1, 1]], color=\"blue\", label=\"e2\")\n",
    "ax.legend()\n",
    "plt.xlim(-3.5, 3.5)\n",
    "plt.ylim(-3.5, 3.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如上图所示：标准坐标系K1中有两个向量e1和e2，这两个向量用以描述一个新的二维空间记做K2，那么原坐标系K1中存在这样的一组向量x11和x12，他们在投射到新坐标系K2后形成新的向量组。在K2坐标系下新的向量组记为x21和x22，则x11与x21同方向且向量的模之比为对应特征值，向量x12与x22同方向且向量的模之比为对应特征值。\n",
    "\n",
    "了解了特征值的几何意义之后我们可以将方阵的特征值特征矩阵相乘的形式展开（注：为了表示方便我们假定矩阵A为对称方阵，那么其特征向量两两正交且模为1，则其特征矩阵的逆矩阵就是其自身的转置）：\n",
    "\n",
    "$$A=[w_{1},w_{2},...,w_{m}]\n",
    "\\begin{bmatrix}\n",
    "\\lambda_{1} & ... & 0\\\\\n",
    "0&\\lambda_{i}&0\\\\\n",
    "0&...&\\lambda_{m}\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "w_{1}^{T}\\\\\n",
    "w_{2}^{T}\\\\\n",
    "...\\\\\n",
    "w_{m}^{T}\n",
    "\\end{bmatrix}=[w_{1},w_{2},...,w_{m}]\n",
    "\\begin{bmatrix}\n",
    "\\lambda_{1}w_{1}^{T}\\\\\n",
    "\\lambda_{2}w_{2}^{T}\\\\\n",
    "...\\\\\n",
    "\\lambda_{m}w_{m}^{T}\n",
    "\\end{bmatrix}=\\lambda_{1}w_{1}w_{1}^{T} + \\lambda_{2}w_{2}w_{2}^{T} + ... + \\lambda_{m}w_{m}w_{m}^{T}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然上面的矩阵A为了方便加了比较强的限定条件（方阵且对称），但是其得到的结论却是可以推广的：一个方阵通过特征值分解的方式可以转化为多个特征值特征向量相乘再加和的形式。在方阵的特征值分解过程中，特征值数值上最大的几项往往占据了全部特征值和的很大一部分，因此上述等式最右边的部分只用挑特征值最大的几项相加就足以表征出原矩阵的大部分特征，这也是矩阵压缩的理论基础之一。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 奇异值分解"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的特征值分解仅仅适用于方阵，对于占矩阵绝大多数的非方阵而言是没有办法应用的。因此在这里就引出了奇异值分解。首先，我们的目标是将矩阵A分解成如下小矩阵相乘的形式：\n",
    "\n",
    "$$A_{m \\times n}=U \\Sigma V^{T}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然矩阵A并不是方阵，不能进行特征值分解，但是我们可以对原矩阵进行一定的转换使其变成一个方阵：\n",
    "\n",
    "$$AA^{T}u_{i}=\\lambda_{i} u_{i} \\rightarrow AA^{T}U=U\\Lambda $$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中的U矩阵就是奇异值分解中的左奇异U矩阵，为什么这么说呢？其实这可以非常容易的证明：\n",
    "\n",
    "$$AA^{T}=U\\Sigma V^{T}V\\Sigma U^{T}=U\\Sigma^{2} U^{T} \\rightarrow AA^{T}U = U\\Sigma^{2}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述证明也直接给出了奇异值对角矩阵和特征值矩阵的关系：\n",
    "\n",
    "$$\\Sigma = \\sqrt{\\Lambda}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同理："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$A^{T}Av_{i}=\\lambda_{i} v_{i} \\rightarrow A^{T}AV=V\\Lambda $$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里的V就是奇异值分解中的右奇异矩阵，证明方法和左奇异矩阵的证明方法是相同的。综上，我们就可以计算出一个非方阵的奇异值分解。奇异值分解可以很好地将一个矩阵的特征描述出来，正因为有这样的特性所以其用途之一就是对图像进行特征提取进而进行压缩。奇异值分解用于对图像的压缩的原理是：通过奇异值分解提取出图像的主要特征（这一点是依靠奇异值大小来完成的，通常奇异值越大其对应的图像特征就越主要），我们通常只需要利用图像的几个主要特征就能够还原一张图片的主要内容，当然，这样的压缩是有损的。\n",
    "\n",
    "$$A_{m \\times n}=U_{m\\times m} \\Sigma_{m\\times n} V^{T}_{n\\times n} \\approx U_{m\\times k} \\Sigma_{k\\times k} V^{T}_{k\\times n}$$\n",
    "\n",
    "接下来，笔者将通过一个例子来说明奇异值分解在图像压缩方面的应用。假定我们要进行压缩的图片如下所示：\n",
    "\n",
    "![](https://upload-images.jianshu.io/upload_images/8762099-0933fc4d649b8349.jpg?imageMogr2/auto-orient/strip%7CimageView2/2/w/960)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于这是一张三通道图片，因此不能直接对其进行奇异值分解。首先要做的是将三个通道的图像分开，然后分别进行奇异值分解，最后将压缩后的三个通道再重叠在一起就能还原出一幅彩色图片。整个程序非常简单：\n",
    "\n",
    "```python\n",
    "def img(img_path, k):\n",
    "    img = cv2.imread(img_path)\n",
    "    height = img.shape[0]\n",
    "    width = img.shape[1]\n",
    "    r = np.asmatrix(img[:, :, 0], dtype=np.float32)\n",
    "    g = np.asmatrix(img[:, :, 1], dtype=np.float32)\n",
    "    b = np.asmatrix(img[:, :, 2], dtype=np.float32)\n",
    "    U1, sigma1, V1T = np.linalg.svd(r)\n",
    "    sigma1 = sigma1[0: k]\n",
    "    U2, sigma2, V2T = np.linalg.svd(g)\n",
    "    sigma2 = sigma2[0: k]\n",
    "    U3, sigma3, V3T = np.linalg.svd(b)\n",
    "    sigma3 = sigma3[0: k]\n",
    "    sigma1 = np.hstack(\n",
    "        (np.vstack((np.diag(sigma1), np.zeros(shape=(height - k, k)))), np.zeros(shape=(height, width - k))))\n",
    "    sigma2 = np.hstack(\n",
    "        (np.vstack((np.diag(sigma2), np.zeros(shape=(height - k, k)))), np.zeros(shape=(height, width - k))))\n",
    "    sigma3 = np.hstack(\n",
    "        (np.vstack((np.diag(sigma3), np.zeros(shape=(height - k, k)))), np.zeros(shape=(height, width - k))))\n",
    "    channel_1 = np.asmatrix(U1 * sigma1 * V1T, dtype=np.uint8)\n",
    "    channel_2 = np.asmatrix(U2 * sigma2 * V2T, dtype=np.uint8)\n",
    "    channel_3 = np.asmatrix(U3 * sigma3 * V3T, dtype=np.uint8)\n",
    "    img = cv2.merge([channel_1, channel_2, channel_3])\n",
    "    cv2.imshow(\"test\", img)\n",
    "    cv2.waitKey(0)\n",
    "    cv2.destroyAllWindows()\n",
    "```\n",
    "\n",
    "现在先来测试比较极端的情况：只取第一个奇异值，剩余的全部置0。这种情况下，图片压缩后的效果为：\n",
    "\n",
    "![](https://upload-images.jianshu.io/upload_images/8762099-5e47c851044bdce9.jpg?imageMogr2/auto-orient/strip%7CimageView2/2/w/960)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到只选取一个奇异值整个图片非常模糊，只能看出图片的大致结构和色调。现在我们选取前5个奇异值得到的效果如下：\n",
    "\n",
    "![](https://upload-images.jianshu.io/upload_images/8762099-76361f38c05502e4.jpg?imageMogr2/auto-orient/strip%7CimageView2/2/w/960)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "选取前50个奇异值得到的结果如下：\n",
    "\n",
    "![](https://upload-images.jianshu.io/upload_images/8762099-e45e7588e959d83d.jpg?imageMogr2/auto-orient/strip%7CimageView2/2/w/960)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，整个图片已经比较清晰了。如果我们不断增加选取的奇异值个数（当然，这里的奇异值是从大到小排过序的）那么获得的图片还原度就越高，当然压缩比例也就会越小。处理之外，奇异值分解还被应用在数据降维等多个方面，在这里就不一一赘述。\n",
    "\n",
    "2019年5月31日"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
